%0 Conference Proceedings
%4 sid.inpe.br/sibgrapi/2023/08.11.19.30
%2 sid.inpe.br/sibgrapi/2023/08.11.19.30.50
%@doi 10.1109/SIBGRAPI59091.2023.10347043
%T Heuristics to reduce linear combinations of activation functions to improve image classification
%D 2023
%A Moraes, Rogério Ferreira de,
%A Evangelista, Raphael dos S.,
%A Pereira, Andre Luiz da S.,
%A Toledo, Yanexis Pupo,
%A Fernandes, Leandro A. F.,
%A Martí, Luis,
%@affiliation Universidade Federal Fluminense (UFF), Niterói, Brazil
%@affiliation Universidade Federal Fluminense (UFF), Niterói, Brazil
%@affiliation Universidade Federal Fluminense (UFF), Niterói, Brazil
%@affiliation Universidade Federal Fluminense (UFF), Niterói, Brazil
%@affiliation Universidade Federal Fluminense (UFF), Niterói, Brazil
%@affiliation Inria Chile Research Center, Las Condes, Chile
%E Clua, Esteban Walter Gonzalez,
%E Körting, Thales Sehn,
%E Paulovich, Fernando Vieira,
%E Feris, Rogerio,
%B Conference on Graphics, Patterns and Images, 36 (SIBGRAPI)
%C Rio Grande, RS
%8 Nov. 06-09, 2023
%S Proceedings
%K learned activation function, trainable activation function, linear combination of activation functions.
%X Image classification is one of the classical problems in computer vision, and CNNs (Convolutional Neural Networks) are widely used for this task. However, the choice of a CNN can vary depending on the chosen dataset. In this context, we have trainable activation functions that are crucial in CNNs and adapt to the data. One technique for constructing these functions is to write them as a linear combination of other activation functions, where the coefficients of this combination are learned during training. However, if we have a large number of activation functions to combine, the computational cost can be very high, and manually testing and choosing these functions may be impractical, depending on the number of available activation functions. To alleviate the difficulty of choosing which activation functions should be part of the linear combination, we propose two heuristics: Linear Combination Approximator by Coefficients (LCAC) and Major and Uniform Coefficient Extractor (MUCE). Our heuristics provide an efficient selection of a subset of activation functions so that their results are better or equivalent to the linear combination that uses all 34 available activation functions in our experiments (C34), considering the image classification problem. Compared to the C34 function, the LCAC function was better or equivalent in 62.5%, and the MUCE function in 87.5% of the conducted experiments.
%@language en
%3 Moraes-paper50.pdf